Error checking technique for use in mass storage systems

ABSTRACT

A system of error checking a sequence of concatenated packets of data for errors utilizes a CRC checker and a predictor which generates a prediction based on an all ones sequence of bytes the same length as the sequence. The sequence includes segments ending with bang CRCs (!CRCs) and padding bytes of all ones.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of Provisional Application SerialNo. 60/049,908, filed Jun. 18, 1997, which is incorporated herein byreference for all purposes.

BACKGROUND OF THE INVENTION

The present invention relates generally to error detection and, moreparticularly, to error detection in a system where data is “packetized”and passes through several interfaces prior to storage on non-volatilemedia.

Cyclic Redundancy Checking (CRC) is a well-known error detectiontechnique that calculates CRC byte(s) for data and appends the CRC bytesto the data to be propagated together as a CRC packet. A CRC checkerthen checks the CRC packet to determine whether the data has beencorrupted during propagation.

Subsequently, the CRC packet itself is grouped with other CRC packets toform “super packets”. It is then necessary to generate CRC bytes for thesuper packet and append those bytes to the super packet to form a “superCRC packet”.

As the super CRC packet is passed through various interfaces, it shouldbe checked to be sure that data is not corrupted.

Generating the CRC bytes necessary for error checking and checking thevarious packets and super packets is computation intensive and consumesvaluable processing resources as well as introducing delays into thepropagation of data.

Accordingly, efficient techniques to assure integrity of data stored onnon-volatile media are needed in the industry.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, error checkingutilizing cyclic redundancy checking (CRC) techniques is utilized tocheck for errors as data passes across multiple interfaces.

According to another aspect of the invention, bang CRCs (!CRC) arepredicted for cumulated data structures by utilizing the properties ofappended bang CRCs using an equivalent structure of all “1”s. CRCs ofcumulative CRC^(C) structures are checked by comparing such predictedresult and the result of explicit byte-by-byte calculation in thechecker.

According to another aspect of the invention, the cumulative CRC^(C) iscalculated by the interface unit coupling a virtual drive emulationsystem to a disk storage unit.

Other features and advantages of the invention will be apparent in viewof the following detailed description and appended drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a preferred embodiment of the invention;

FIG. 2 is a schematic diagram depicting the data paths of the systemdepicted in FIG. 1;

FIG. 3 is a schematic diagram of an IDRC packet;

FIG. 4 is a schematic diagram of a CRC checker; and

FIG. 5 is a pseudo-code chart depicting a CRC prediction method.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A preferred embodiment will now be described with reference to thefigures, where like or similar elements are designated with the samereference numerals throughout the several views. FIG. 1 is a high levelblock diagram of a part of a tape drive emulation (TDE) system 10utilizing an embodiment of the present invention.

A plurality of channel interfaces (CIFs) 12 are coupled to host I/Ochannels (not shown) to transfer data between the host and the TDEsystem 10.

Each CIF 12 includes a host interface 14, an embedded controller 16, adata formatter 18 for performing data compression and other functions,an SBUS interface 20, a buffer memory 22, and an internal bus 24. In thepreferred embodiment, the embedded processor 16 is a model i960manufactured by Intel Corporation.

The main controller 30 includes a main processor 32, main memory 34, anSBUS interface 36, and an internal bus 38. In the preferred embodiment,the main processor is a SPARC computer manufactured by Sun MicrosystemsIncorporated. The CIFs 12 and main controller 30 are coupled by a systembus 40.

The tape drive emulation (TDE) system 10 stores host data in “virtualvolumes” mounted on “virtual tape drives.” In one preferred embodiment,the data is originally stored on staging disks 60. Because TDE system 10must interact with the host as if the data were actually stored on tapedrives, a data structure called virtual tape drive (VTD) is maintainedin main memory 34 for each virtual tape drive. The VTD containsinformation about the state of the associated virtual tape drive.Additional structures, including a virtual tape “volume” structure andother structures subordinate to it, register the locations at which datais physically stored, among other information. The CIFs 12 areimplemented on dual system boards (DSBs) in a preferred embodiment.

Subsequently, data may be transferred from the staging disks 60 to oneor more magnetic tape units 62. However, the location and otherproperties of the data is still defined in terms of the virtual tapevolume structures in memory and stored in a disk-based control data set.

The data paths of the embodiment depicted in FIG. 1 are shown in FIG. 2.In the preferred embodiment, data is first stored on staging disks 60.The staging disk space is treated as collections, called regions, offixed-size space units called extents.

Referring to FIG. 2, a host computer coupled to a TDE system isdepicted. A data I/O channel (not shown) of the host is coupled to anESCON port, one of several which function as a network to connectseveral host channels and possibly several hosts to the TDE system.

The ESCON port is a component of a channel interface (CIF) 12 includingbuffers 70 and 72 and the data compression unit 18. The CIF 12 iscoupled to an extent buffer 74 and the main controller 30 by the SBUS40. The main controller 30 is coupled to the staging disks 60 (which inthis embodiment is a RAID) by a first host adapter (HA) 76 and RAIDcontroller 78 and to a magnetic tape unit (MTU) 62 by a second HA 80 anda magnetic tape controller unit 82 (MCU).

Packets are formed in the CIFs 12 and stored in extent buffer 74. In apreferred embodiment, the packets are packetized on the model of datapackets of the 3480/IDRC or 3490 (see ANSI X3.224-1994, X3.225-1994).

Each data block written to the TDE system is compressed (if possible andunless a channel command prevents it) in hardware of the channelinterface and “packetized” on the model of data packets of the 3480/IDRCor 3490 (see ANSI X3.224-1994, X3.225-1994). A block is compressedindependently of all other blocks; that is, the compression algorithm isreset at the beginning of each block and any single block can bedecompressed by itself.

The packet includes the byte sequence output by the compressor (or theuncompressed block byte sequence if no-compression is invoked); afixed-length preamble, variously called the Packet-ID or the PacketHeader; and, a postamble of variable length called the Packet Trailer.The combined length of the data and the Packet Trailer is constrained inthe preferred embodiment to be a multiple of 32 bytes. Padding (inserted0's) is included in the trailer to satisfy this constraint.

FIG. 3 shows details of the packet formats for compressed anduncompressed blocks, respectively. Each packet formed in a CIF isassigned a block ID which is included in the packet. Packet header bytesfrom offset 113 through offset 29 are required by the format standard tobe filled with the value 00H. The channel interface may store a“signature” of several bytes in this field for use in integrity checksand diagnostics. The bytes will be cleared, however, before the packetis written to tape. See FIG. 3 for definitions of the bytes. This formatis well-known in the art and is not part of the present invention.

The CIF buffers 70 and 72 are regions of the channel interface memory20. The extent buffer 74 is a region of main memory 34. The size of theextent buffer is the same as the fixed-space unit on the disk tofacilitate efficient transfer between the staging disks 60 and extentbuffer 74.

In one write mode of operation, entire packets are transferred from theCIFs 12 to the extent buffer 72. When there is not room in the extentbuffer for another entire packet the remaining unused part of the extentbuffer is packed with ones and then the contents of the buffer arewritten to a fixed-sized region on the staging disk.

Prior to describing the operation of the present preferred embodiment, abrief overview of CRC calculation and checking is presented. Thesetechniques are well known in the art, and are described in more detailin appendix A.

Briefly, a sequence of data bytes [D1, D2, D3, . . . Dn] is representedas polynomials with powers of x indicating the position of a data bit inthe sequence and the coefficients of the power of x being the value ofthe bit.

Thus, for example, each byte Di is represented by the polynomial:

Di=d _(i1) x ⁷ +d _(i2) x ⁶ +. . . d _(i7) x+d _(i8).

The value of the most significant bit in the byte is d_(i1) and thevalue of the least significant bit is d_(i8).

The entire byte sequence then is represented by a large polynomial,D(x):

D(x)=D ₁ x ^(8(n−1)) +D ₂ x ^(8(n−2)) +. . . +D _(n)

When polynomial are added or subtracted, co-efficient arithmetic ismodulo-2 arithmetic for coefficients of like powers of x.

After CRC bytes are calculated and appended to the data sequence, anextended sequence is formed:

E=[D1, D2, . . . Dn, C1, C2]

and in polynomial form:

E(x)=D(x)x ¹⁶ +C1x ⁸ +C2.

The CRC bytes are chosen to have the property that:

E(x)=0, modulo-G(x), where

G(x)=x ¹⁶ +x ¹⁵ +x+1=0.

A simple CRC generator is depicted in FIG. 4.

Additionally, a “bang CRC”, denoted !CRC, is calculated by firstinverting the bits in the data sequence, calculating the CRC bytes, asdescribed above, on the inverted data sequence, and then inverting thecalculated CRC bytes themselves.

If the bang CRC bytes, calculated from the inverted data sequence, areappended to the original, non-inverted, data sequence, then an extendedbang sequence is formed:

EB=[D1, D2, . . . Dn, !C1, !C2].

This extended bang sequence has the useful property that whenpolynomialized and divided by G(x), the remainder is the same asdividing a sequence of all ones, having the same length as EB, by G(x).

As depicted in FIG. 3, the packet header is in the form of an extendedbang sequence, as is the concatenation of the compressed user data andpacket trailer. Another useful property of the extended bang sequencesis that if two EBs are concatenated, the remainder of division by G(x)is the same as the remainder of a sequence of all ones bytes, equal inlength to the concatenated bang sequence, divided by G(x).

Returning now to the description of the operation of the presentpreferred embodiment, in one write mode of operation, a shortdescriptive extent header, including a bang CRC covering the informationbytes of the header, is placed at the beginning of an extent buffer.Thereafter, entire packets are transferred from the CIFs 12 to theextent buffer 72. When there is not room in the extent buffer foranother entire packet, the remaining unused part of the extent buffer ispacked with ones and then the contents of the buffer, termed an extent,are written to a fixed-sized region on the staging disk. Alternatively,only part of the remaining part of the buffer may be padded with ones,in the smallest amount needed to make the aggregate length of header,packets, and pad bytes commensurate with the sector size of the disk,and that aggregate length is written to the disk.

Thus, the extent is a concatenation of the header, the IDRC packets, anda sequence of all ones bytes. Another useful property of the extendedbang sequences is that if a padding sequence of all ones bytes isconcatenated with a concatenated sequence of extended bang sequences,the remainder of division by G(x) is the same as the remainder of asequence of all ones bytes having the same length as the concatenatedsequence of extended bang sequences concatenated with the paddingsequence of all ones bytes.

As described above with reference to FIG. 2, data is transferred fromthe extent buffers 74 to staging disks 60 through the first host adapter76 and RAID controller 78 and subsequently, in some cases, from stagingdisks through extent buffers and to the tape drives 62 through thesecond host adapter 80 and the MCU 82.

The present invention provides a high degree of data integritymonitoring by performing CRC checking as it passes through each hostadaptor 76 and 80 and the tape controller 82. Additionally, the need tocompute CRC bytes explicitly by operating on the whole collection ofbytes stored in the buffer for each extent formed in the extent bufferis eliminated by the unique system of error checking described below.

In the presently described embodiment, the first host adapter 76interfaces between the SBUS 40 and a SCSI bus 100. When an extent (ornumber of extents) is transferred from the extent buffer 74 to the RAIDcontroller, it is error checked in first host adapter 76. A checkingvalue is obtained by dividing the sequence of data bytes in the extentby G(x), using a calculating circuit provided on the host adapter forthe purpose. The checking value is compared to a value predicted bysoftware in the SPARC processor, based only on the length of thetransfer.

In the presently described embodiment, a first CRC checker 200 is builtinto the SCSI port of the first HA 76 to which staging disk controller78 is attached, and a second CRC checker 220 is built into the SCSI portof the second HA 80, to which the MCU 80 is attached.

The CRC checker is conceived as either a SCSI target installed on theSBUS card and attaching to the SCSI-bus side of the first HA 76 or as apassive “snooper” circuit attaching to the SCSI-bus side and controlledby the HA. The CRC checker monitors data outbound to any, or any other,target on the SCSI bus. If desired, it also monitors data inbound.

The CRC checker computes a CRC value for the data transfers of allwrites to a given target or Logical Unit (LUN) when a target supportsmultiple LUNs. For multiple targets or LTJNS, separate CRC values foreach target or LUN may be computed. In an alternative supported mode ofoperation, separate CRCs are calculated for each write command issuedwhen the optional SCSI feature of tagged command queuing is in use. Theonly command that must be supported by the checker is the reading of thepresent state (value) of the CRC checker for a particular target. Auseful alternative mode of operation is provided if the calculatorcontrol firmware can be enabled to initialize the value uniformly to adesired starting value at the start of each write. In another extension,the circuit is used to calculate CRCs on data being read from the disk.In all cases, either the HA control program (if a passive snooper isused) or the checker firmware (if the checker is a separate SCSI target)captures the states of the calculator between operations to make themavailable to the SPARC software.

When the checker is to be used in the mode afforded when the calculatoris read-only, the SPARC-resident software reads the state of thecalculator before the beginning of one or a sequence of several writesand reads it again on the completion of the sequence. The finalcalculator state is predicted by the software from the initial state andknowledge of the aggregate length of written extent data. In the mode inwhich the calculator is initialized for each operation to a uniformstarting state, only the end state must be read.

Pseudo-code for a technique of generating a new predicted value from anold one and a knowledge of the total length of transfers interveningbetween value readings is set forth in FIG. 5. These prediction stepsare performed by software executed in the main processor 32. Referringto FIG. 5, the prediction is generated utilizing three tables. If theold prediction is 0000H, then steps 1 and 2 may be skipped, leaving0000H as the partial prediction. In step 1, the first table is alogarithm table organized as 65,535 16-bit words, each containing anelement in the finite (Galois) field of 2¹⁶ elements. Otherwise, indexinto the table and save the logarithm value fetched for step 2.

In step 2, the second table is organized as an antilog table having alength of 65,535 16-bit words. To index the second table, add the logvalue obtained in step 1 to the aggregate length (counted in 16-bitwords) of all extents written since the last prediction update. Reducethe sum modulo-65,535, that is, divide the sum by 65,535, throw away thequotient and keep the remainder, and index the second table by theresult to find a new partial prediction.

In step 3, the third table is indexed by the aggregate length of allextents which have been written through the SCSI port to this LUN sincethe last update. The length is measured in words of sixteen bits and isreduced modulo-65,535 before its use as an index. Add, bitwise modulo-2,the value from the third table to the partial prediction. This yieldsthe new prediction which is saved and compared to the state of thechecker.

Techniques for generating the first, second, and third tables are setforth in appendix A.

The invention has now been described with reference to the preferredembodiments. Alternatives and substitutions will now be apparent topersons of skill in the art. In particular, the packet format depictedin FIG. 3 may be modified as long as each part of the packet or thewhole packet is in the form of an extended bang sequence. Similarly,byte groups other than packets may be inserted in the sequence as longas each of such byte groups is in the form of an extended bang sequence.The particular form of the CRC checker and algorithm for predicting CRCmay be modified in ways well known to those skilled in the art.Accordingly, it is not intended to limit the invention, except asprovided by the appended claims.

APPENDIX

8.1 A CRC GENERATOR

(1) G(x)=x¹⁶+x¹⁵+x⁸+x+1 The “Generator Polynomial”

(2) Call the sequence of data bytes received at the CRC calculator(block size=n)

D=[D₁, D₂, . . . D_(n)]

(ANSI byte numbering convention)

(3) After CRC bytes are calculated and appended to the data string, theresult is:

E=[D₁, D₂, . . . D_(n), C₁, C₂]

(The choice of ‘E’ here is suggestive, because E is ‘extended’ from byadding the two bytes C₁, C₂.)

If the bytes in the sequence are viewed as groups of coefficients in apolynomial, that is, each D₁=d_(i1) x⁷+d_(i2)x⁶+. . . +d_(i7)x+d_(i8)(d_(ij) are the bits in the byte), then the whole string may be viewedas a polynomial: $\begin{matrix}{{E(x)} = \quad {{D_{1}x^{{8{({n - 1})}} + 16}} + {D_{2}x^{{8{({n - 2})}} + 16}} + \ldots + {D_{n}x^{16}} + {C_{1}x^{8}} + C_{2}}} \\{= \quad {{{D(x)} \cdot x^{16}} + {C_{1}x^{8}} + C_{2}}}\end{matrix}$

(4) If C₁ and C₂ have been chosen properly, the polynomial C(x) has theproperty that

E(x)=0, modulo-G(x)⁹, or

D(x)·x ¹⁶ =C ₁ x ⁸ +C ₂, modulo-G(x)

That is, if C(x) is divided by G(x) (using modulo-2 arithmetic on thecoefficients), the division is exact, and the remainder is 0.

(5) To find the right values for C₁ and C₂, we may calculate themsuccessively as data bytes arrive, looking up values in a table whichcontains all 256 values of D·x¹⁶, modulo-G(x): $\begin{matrix}{{{D_{1}x^{16}} = \quad {{C_{1,1}x^{8}} + C_{2,1}}},{{table}\quad {at}\quad D_{1}}} \\{{{{D_{1}x^{24}} + {D_{2}x^{16}}} = \quad {{C_{1,2}x^{8}} + C_{2,2}}},{{{table}\quad {at}\quad C_{1,1}} \oplus D_{2}},} \\{\quad {C_{2,1}\quad {added}\quad {to}\quad {MSB}\quad {to}\quad {get}\quad C_{1,2}}} \\{{{{D_{1}x^{32}} + {D_{2}x^{24}} + {D_{3}x^{16}}} = \quad {{C_{1,3}x^{8}} + C_{2,3}}},{{{table}\quad {at}\quad C_{1,2}} \oplus D_{3}},} \\{\quad {C_{2,2}\quad {added}\quad {to}\quad {MSB}\quad {to}\quad {get}\quad C_{1,3}}} \\{\ldots \quad \ldots} \\{{{\ldots + {D_{n}x^{16}}} = \quad {{C_{1,n}x^{8}} + C_{2,n}}},{``\quad {``{{C_{1,{n - 1}} \oplus D_{n}},{C_{2,{n - 1}}{``\quad {``\quad {``\quad {``\quad {``C_{1,n}}}}}}}}}}}\end{matrix}$

At each step, we multiply the previous CRC by x⁸, equivalent tomultiplying the polynomial of data bytes so far by x⁸, combine with thenew data byte (D_(new)x¹⁶), and reduce the result modulo-G(x).

(6) FIG. 1 shows a concept diagram of a CRC calculator.

⁹The meaning of “modulo-G(x)” is that in the algebra of thesecalculations. G(x) = 0, that is, x¹⁶ + x¹⁵ + x⁸ + x + 1 = 0, or   x¹⁶ =x¹⁵ + x⁸ + x + 1 (remember that coefficient arithmetic is modulo-2) sothat any polynomial of degree sixteen or greater can be reduced to oneof degree less than sixteen “modulo-G(x).”

8.2 THE CRC'S IN EDRC

(1) ANSI X3.225 specifies that some CRC's are to be formed by invertingthe data before the CRC calculation and then inverting the resulting CRCbefore appending it to the data. With the aid of some special tables,software can convert an ordinary CRC to the value it would have underthis ‘inverting’ prescription.

Let the all-ones byte (hex FF) be

U=x ⁷ +x ⁶ +x ⁵ +x ⁴ +x ³ +x ² +x+1

The encoding of i all-ones bytes with CRC appended is:

U _(i)(x)=Ux ^(8(i−1)+16) +Ux ^(8(i−2)+16) +. . . +Ux ¹⁶ +C _(1u) x ⁸ +C_(2u)

The ordinary encoding of an n-byte data string to n+2 bytes includingthe CRC is:

E(x)=D ₁ x ^(8(n−1)+16) +D ₂ x ^(8(n−2)+16) +. . . +D _(n) x ¹⁶ +C ₁ x ⁸+C ₂

And the ‘inverted’ encoding of the n-byte string is (“{circumflex over ()}” means inversion or complementing)

{circumflex over ( )}E′(x)={circumflex over ( )}D ₁ x ^(8(n−1)+16)+{circumflex over ( )}D ₂ x ^(8(n−2)+16) +. . . +{circumflex over ( )}D_(n) x ¹⁶ +C ₁ ′x ⁸ +C ₂′

Remembering that arithmetic on coefficients here is modulo-2, that is,that “+” means XOR, you can see that

{circumflex over ( )}D _(i) =D _(i) +U

I  say, without  proving  it, that  these  polynomials  preserve  theirinteresting  properties  when  they  are  combined  in  such  linearoperations  as  addition  of  the  polynomials. For  instance,E_(A)(x) + E_(B)(x) = (D_(1A) + D_(1B))x^(8(n − 1) + 16) + (D_(2A) + D_(2B))x^(8(n − 2) + 16) + … + (D_(n  A) + D_(nB))x¹⁶ + (C_(1A) + C_(1B))x⁸ + (C_(2A) + C_(2B))       In  particular,                                (x) = E(x) + U_(n)(x)from  which,C₁^(′)=C₁+C_(1u),C₂^(′)=C₂+C_(2u),(C_(1u),C_(2u)  looked  up in  a  table)  and  the  CRC  required  by  the  ‘inverting’  prescriptionis  found  by  inverting  C₁^(′)  and  C₂^(′)Further, the  inverting  of  the  calculated  CRC  is  equivalent  toadding  Ux⁸+U(16  ones)  to  C₁^(′)x⁸+C₂^(′)Remember  (1)  that  we  only  invert  the  data  to  calculate  theCRC, i.e., we  store  the  data  D(x)  without  inversion, (2) that  wealso  store  the  CRC  compounded  from(C₁x⁸ + C₂) + (C_(1u)x⁸ + C_(2u)) + (Ux⁸ + U)and  (3)  in  a  subsequent  division  of  the  complete  stored  polynomial(by  a  CRC  checker, for  example),  the  contribution  to  the  result  byE(x) =D(x) +C_(  1)x⁸+C₂  is  always  0  sinceD(x) = (C₁x⁸ + C₂), modulo  G(x).Therefore, to  understand  the  checker’s  calculation, we  only  need  tothink  about  the  division  of$\underset{\_}{\left( {{C_{1u}x^{8}} + C_{2u}} \right) + \left( {{Ux}^{8} + U} \right)}\quad {by}\quad {{G(x)}.\text{Since~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}}$U_(i)(x) = Ux^(8(i − 1) + 16) + Ux^(8(i − 2) + 16) + … + Ux¹⁶ + C_(1u)x⁸ + C_(2u), modulo  G(x)Ux^(8(i − 1) + 16) + Ux^(8(i − 2) + 16) + … + Ux¹⁶ + 0x⁸ + 0 = C_(1u)x⁸ + C_(2u), and${{Ux}^{{8{({i - 1})}} + 16} + {Ux}^{{8{({i - 2})}} + 16} + \ldots + {Ux}^{16} + {Ux}^{8} + U} = {\underset{\_}{\left( {{C_{1u}x^{8}} + C_{2u}} \right) + \left( {{Ux}^{8} + U} \right)}.\text{In~~other~~words, the~~checker~~will~~produce~~the same~~result~~in~~it's~~division~~whether~~it~~divides~~the~~stored~~polynomial~~or~~a~~sequenceof~~ones~~having~~the~~same~~total~~length~~as~~the~~stored~~polynomial!}}$

2) Figure shows the contents of the header of an EDRC Packet. Figure andFigure show the user data and the packet trailer fields for packet datawhich does and does not compress. In the OSS emulation of EDRC, thePacket header CRC, CRC1, can be computed by software executing thealgorithm of Figure, after the other fields have been filled in. Thedata bytes must be inverted before the calculation and the CRC invertedafterward.

Alternatively, the Figure algorithm can be used to compute a CRC on the(inverted) first 13 bytes and the insertion of the 17 pad bytes (bytes14-30) can be computed.

This can be done with the aid of two more 256×16-bit lookup tables, eachindexed by one of the CRC bytes calculated on the first 13 bytes. Onetable contains the results of multiplying all possible byte values byx⁸⁺⁸⁽¹⁷⁾ and the other the results for multiplication by x⁸⁽¹⁷⁾. Part ofthe wanted CRC is found by XOR′ing the result of the two lookups. Thebasis for this calculation is that

D(x)·x ¹⁶ =C _(1a) x ⁸ +C _(2a), modulo-G(x),

where D(x) is the ‘polynomialized’ string of the first 13 bytes

Shifting left to make room for the pad bytes is done asD(x) ⋅ x¹⁶ ⋅ x^(8(17)) = (C_(1a)x⁸ + C_(2a))x^(8(17)) = C_(1a)x^(8 + 8(17)) + C_(2a)x^(8(17))

The lookups give the modulo-G(x) values of the two terms on the right.These (16-bit) values must be XOR′ed with each other and with a fixedvalue for a string of 17 all-ones-bytes (the inverted padding). Finally,the result must be inverted.

Figure 21. Packet-ID (Packet Header) Contents Bytes: 1 2 3-6 7-10 11 1213 14-30 31.32 80H prv block-# packet trlr comp alg 00...0 pkt-id lengthlgth flag # CRC (id+user) “CRC1”

(3) Hardware must be included to compute a CRC (CRC-u=C_(u1)x⁸+C_(u2))on the uncompressed data and a CRC (CRC-c=C_(c1)x⁸+C_(c2)) on thecompressed data If the data is found to be compressible, that is, if itdoes not expand in the compression attempt, then the value in CRC2 isCRC-u.

FIG. 22. Compressed User Data and Packet Trailer B₁ = B_(n) <== T₁ = T₆==> (size (n) (4) (2) (2) (m) (2) (A: compr. uncomp uncomp CRC, pad CRC,processed user data data user + (00 . . . 0) user + data case) data lgthCRC trlr trlr “CRC2” (1-6) (=>pad) “CRC3” “CRC4”

(4) The value of CRC3 can be found with software. Since the CRCcalculation on the compressed data givesB₁x^(8(n − 1) + 16) + B₂x^(8(n − 2) + 16) + … + B_(n)x¹⁶ = C_(c1)x⁸ + C_(c2), modulo − G(x)

We can “extend” CRC-c to make room for the first six trailer bytes bymultiplying

(C _(c1) x ⁸ +C _(c2))x ⁸⁽⁶⁾ =C _(c1) x ⁸⁽⁷⁾ +C _(c2) x ⁸⁽⁶⁾,modulo-G(x)

and insert the trailer bytes, T₁-T₆ as

T₁x¹⁶⁺⁸⁽⁵⁾+. . . T₆x¹⁶, modulo-G(x).

We can compute by first XOR′ing T₁ with C_(c1) and T₂ with C_(c2), sincethese are coefficients of x⁸⁽⁷⁾ and x⁸⁽⁶⁾, respectively, and thenapplying the FIG. 1 algorithm on the six bytes:

(T₁+C_(c1)), (T₂+C_(c2)), T₃, T₄, T₅, T₆,

to find the bytes of CRC3.

(5) CRC4 can be computed from CRC3. When CRC3 is appended to (compresseduser data+trailer bytes 1-6, making a trailer of eight bytes length, aCRC calculated on the entire new sequence has the value 0, identically.Therefore, continuing to calculate as pad bytes are appended givessuccessively more values of 0 (0x⁸+0). Applying the ‘inversion’ schemeof 8.2 (2) yields a CRC, (before the final inversion) which is just theCRC on (n+4+2+2+m) all-ones bytes and therefore only depends on the datalength.

FIG. 23. Uncompressed User Data and Packet Trailer (size (n′) (m′) (2)(B: non- uncompr. pad !CRC processed user (00 . . . 0) user + data case)data trlr (=>pad) “CRC4”

(6) For data which does not compress, the Packet Trailer is as shown inFigure. For this case, CRC4 can be computed from CRC-u, the CRCcalculated by hardware on the uncompressed data. The calculation done bythe hardware can be continued in software to append the “m” pad bytes,or table lookup could be done to multiply the bytes of CRC-u by x^(8(m))and x^(8(m−1)), respectively. Thirty-one pairs of 256×16-bit tableswould be needed for all nonzero values of m. The effect of datainversion might be computed by the method of 8.2 (1).

8.3 CRC's on Packet Sequences

In the box in section 8.2, we established (or at least declared) that aCRC checker doing division on a stored span of data including one of the!CRC's gets the same result as if it did the division on a string of allones of the same length as the span. Routing a packet (data+trailer oflength n) to the checker immediately following a packet header isequivalent to concatenating the two in this way:

E_(packet hdr)(x)*x^(8(n+1))+E_(packetdata & trir)(x),

and that is equivalent to routing to the checker a string of all-onesbytes of length n+32, the combined length of the packet.

Similarly, routing two packets (lengths n and m) in unbroken sequence toa checker concatenates them as

E_(1st packet)(x)*x^(8(m+1))+E_(2nd packet)(x),

and is equivalent to routing (n+m) all-ones bytes to the checker.

This equivalence to ‘length-of-ones’ extends to concatenations of anynumber of packets or other structures containing a !CRC. And of course,actual strings of ones can be inserted between packets and their lengthsbe added to the aggregate length in the equivalence assertion.

8.4 The SCSI-Port CRC Checker

We review here the idea and use of a CRC checker built into the SCSIports to which staging disk controllers are attached.

The CRC checker is conceived as a SCSI target installed on the S-Buscard and attaching to the SCSI-bus side of the interface. The checkermonitors all data outbound to any other target on the bus.

1. The CRC checker computes a CRC value for the data transfers of allwrites to a given target. If multiple targets are attached, the checkermust have a separate CRC value for each, possibly accessible as aseparate logical unit within the checker target, numbered to reflect thenumbering of the monitored targets.

2. CRC's calculated for targets which are not RAID controllers can beignored for our purposes.

3. The only command that must be supported by the checker is the readingof the present state (value) of the CRC checker for a particular target.

4. As I/O is directed to a RAID target, someone (possibly the devicedriver) will maintain an ongoing prediction of the state of the CRCchecker.

5. Each time synchronization is required, that is, each time in thewriting of a virtual volume that a Write Tape Mark, Synchronize, or ReadBackward command or a tape motion command is issued, the driver willread the CRC checker's state after the last data of the virtual volume(to that point) has been written and will compare the value read againstthe value predicted. A match is required for the synchronization tosucceed.

6. Testing of the checker state could and probably should be done morefrequently, but the test must be done at synchronization points toconfirm that data up to those points has been sent correctly to thedisk.

7. A preferable schedule of checker testing would be at the completionof each sequence of write commands done on behalf of a particularvirtual tape unit. The most practical choice might be to test afterevery write command.

8. The values found in the checker at these readings will not ordinarilybe zeros.

9. The CRC's included by the channel interface at the end of each packetheader and each packet trailer should be the !CRC as prescribed by theIDRC standard, and not the “normal” CRC proposed earlier by me for dataon its way to the staging disk.

The checker is presumed to check separately write transfers to differentdisk drives (arrays) attached to it. A separate prediction is needed foreach such logical device. Predictions of the sixteen-bit CRC checkerstate requires some calculations by the staging disk scheduler or driver(or ??) program executing in the SPARC machine. In particular, each timean updated prediction is needed, the old prediction must be changed byadding (⊕) to it values found with the aid of three tables:

1. The first table is indexed by the old prediction. This table is alogarithm table organized as 65.535 16-bit words, each containing thelogarithm of an element in the finite (galois) field of length 2¹⁶. Ifthe old prediction is 0000H, skip step 2. Otherwise index into the tableand save the value fetched for the next step.

2. The second table is an antilog table of length 65,535, 16-bit words.To index it, add the log from step 1 to the aggregate length (counted in16-bit words) of all extents written since the last prediction update.Reduce the sum modulo-65,535¹⁰ and index the table by the result. Add(⊕) the value from the table to the prediction.

¹⁰This means divide the sum by 65.535, throw away the quotient and keepthe remainder.

3. The third table is indexed by the aggregate length of all extentswhich have been written through the SCSI port to this target drive(array). The length is measured in words of sixteen bits and is reducedmodulo-65,535 before its use as an index. Add (⊕) the value from thetable to the prediction. This yields the new prediction. Save it andcompare it to the state of the checker.

The tables required total≈192 kbytes.

static unsigned short ones_crc[ ]; // see 8.5 for creation of this arraystatic unsigned short crc_log[ ]; // see 8.5 for creation of this arraystatic unsigned short crc_log[ ]; // see 8.5 for creation of this arrayunsigned short predict(prediction, length) unsigned short prediction;unsigned length; // aggregate length since last prediction {length=length%65535; // reduce length mod 2¹⁶ −1 if(prediction) // dosteps 1, 2 prediction=prediction{circumflex over ()}crc_antilog[(crc_log[prediction ]+ length)%65535];prediction=prediction{circumflex over ( )}ones_crc[length]; return(prediction); {

The benefits of this approach to checking:

Implicitly measures the length of transfer: the value of the checker atthe end of each item cycles through the CRC circuit as the next item istransferred, producing a different effect for each of 64k differentshift lengths (16-bit-word shifts). Further, the !CRC incorporates alength-dependent value representing the item length of all-ones values.

Does a CRC check over all data.

Is not defeated by a “stuck” bus: a bus that transmits only all-ones orall-zeroes (or all of any other bit pattern, for that matter will notfool this checker.

8.5 ARITHMETIC WITH THIS GENERATOR POLYNOMIAL

We said that algebra modulo-G(x), (G(x) is the “Generator Polynomial”)means that

G(x)=0.

or

x ¹⁶ +x ¹⁵ +x ⁸ +x+1=0

This implies that

x ¹⁶ =x ¹⁵ +x ⁸ +x+1

The way we apply this idea to a polynomial of degree higher than 15 isto imagine that each non-zero term of high degree is found by successivemultiplications by x. For example (terms with 0 coefficient are notshown, but their presence is implied e.g.,G(x)=1x¹⁶+1x¹⁵+0x¹⁴+0x¹³+0x¹²+0x¹¹+0x¹⁰+0x⁹+1x⁸+0x⁷+0x⁶+0x⁵+0x⁴+0x³+0x²+1x+1),$\begin{matrix}{x^{19} = \quad {x \cdot x \cdot x \cdot x^{16}}} \\{= \quad {x \cdot x \cdot x \cdot \left( {x^{15} + x^{8} + x + 1} \right)}} \\{= \quad {x \cdot x \cdot \left( {x^{16} + x^{9} + x^{2} + x} \right)}} \\{= \quad {x \cdot x \cdot \left( {\left( {x^{15} + x^{8} + x + 1} \right) + \left( {x^{9} + x^{2} + x} \right)} \right)}} \\{= \quad {x \cdot x \cdot \left( {x^{15} + x^{9} + x^{8} + x^{2} + 1} \right)}} \\{= \quad {x \cdot \left( {x^{16} + x^{10} + x^{9} + x^{3} + x} \right)}} \\{= \quad {x \cdot \left( {\left( {x^{15} + x^{8} + x + 1} \right) + \left( {x^{10} + x^{9} + x^{3} + x} \right)} \right.}} \\{= \quad {x \cdot \left( {x^{15} + x^{10} + x^{9} + x^{8} + x^{3} + 1} \right)}} \\{= \quad {x^{16} + x^{11} + x^{10} + x^{9} + x^{4} + x}} \\{= \quad {\left( {x^{15} + x^{8} + x + 1} \right) + \left( {x^{11} + x^{10} + x^{9} + x^{4} + x} \right)}} \\{\underset{\_}{x^{19}} = \quad \underset{\_}{x^{15} + x^{11} + x^{10} + x^{9} + x^{8} + x^{4} + 1}}\end{matrix}$

We can tabulate this process, listing only the coefficients:

term x¹⁵ x¹⁴ x¹³ x¹² x¹¹ x¹⁰ x⁹ x⁸ x⁷ x⁶ x⁵ x⁴ x³ x² x 1 (HEX) x¹⁶ 1 0 00 0 0 0 1 0 0 0 0 0 0 1 1 8103 x¹⁷ 1 0 0 0 0 0 1 1 0 0 0 0 0 1 0 1 8305x¹⁸ 1 0 0 0 0 1 1 1 0 0 0 0 1 0 0 1 8709 x¹⁹ 1 0 0 0 1 1 1 1 0 0 0 1 0 00 1 8F11 x²⁰ 1 0 0 1 1 1 1 1 0 0 1 0 0 0 0 1 9F21 x²¹ 1 0 1 1 1 1 1 1 01 0 0 0 0 0 1 BF41 x²² 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 1 FF81 x²³ 0 1 1 11 1 1 0 0 0 0 0 0 0 0 1 7E01 x²⁴ 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 0 FC02x²⁵ 0 1 1 1 1 0 0 1 0 0 0 0 0 1 1 1 7907 x²⁶ 1 1 1 1 0 0 1 0 0 0 0 0 1 11 0 F20E x²⁷ 0 1 1 0 0 1 0 1 0 0 0 1 1 1 1 1 651F x²⁸ 1 1 0 0 1 0 1 0 00 1 1 1 1 1 0 CA3E x²⁹ 0 0 0 1 0 1 0 1 0 1 1 1 1 1 1 1 157F x³⁰ 0 0 1 01 0 1 0 1 1 1 1 1 1 1 0 2AFE x³¹ 0 1 0 1 0 1 0 1 1 1 1 1 1 1 0 0 55FC

To find some expression of more than one term, we can add (modulo-2)these rows of coefficients. For example, to find x²⁴+x²¹+x¹⁷.$\begin{matrix}x^{24} & \quad & 1 & 1 & 1 & 1 & \quad & 1 & 1 & 0 & 0 & \quad & 0 & 0 & 0 & 0 & \quad & 0 & 0 & 1 & 0 & \quad & {FC02} \\x^{21} & \quad & 1 & 0 & 1 & 1 & \quad & 1 & 1 & 1 & 1 & \quad & 0 & 1 & 0 & 0 & \quad & 0 & 0 & 0 & 1 & \quad & {BF41} \\x^{17} & \quad & 1 & 0 & 0 & 0 & \quad & 0 & 0 & 1 & 1 & \quad & 0 & 0 & 0 & 0 & \quad & 0 & 1 & 0 & 1 & \quad & 8305 \\\quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad & \quad \\{{Sum}:} & \quad & 1 & 1 & 0 & 0 & \quad & 0 & 0 & 0 & 0 & \quad & 0 & 1 & 0 & 0 & \quad & 0 & 1 & 1 & 0 & \quad & {C046}\end{matrix}$

Reorder the table (invert the order of the rows) to find asometimes-useful matrix M:${\lbrack d\rbrack \cdot \quad {\lbrack M\rbrack \left\lbrack {d_{15}d_{14}\quad \ldots \quad d_{0}} \right\rbrack} \cdot \begin{bmatrix}0 & 1 & 0 & 1 & 0 & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 \\0 & 0 & 1 & 0 & 1 & 0 & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 0 \\0 & 0 & 0 & 1 & 0 & 1 & 0 & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 1 & 1 & 1 & 1 & 1 & 0 \\0 & 1 & 1 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 & 0 \\0 & 1 & 1 & 1 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\0 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 \\1 & 0 & 0 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 1 \\1 & 0 & 0 & 0 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 1 \\1 & 0 & 0 & 0 & 0 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 \\1 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 \\1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1\end{bmatrix}}\quad \begin{matrix}{55{FC}} & \quad & x^{31} \\{2{AFE}} & \quad & x^{30} \\{157F} & \quad & x^{29} \\{CA3E} & \quad & x^{28} \\{651F} & \quad & x^{27} \\{F20E} & \quad & x^{26} \\7907 & \quad & x^{25} \\{FC02} & \quad & x^{24} \\{7{E01}} & \quad & x^{23} \\{FF81} & \quad & x^{22} \\{BF41} & \quad & x^{21} \\{9{F21}} & \quad & x^{20} \\{8{F11}} & \quad & x^{19} \\8709 & \quad & x^{18} \\8305 & \quad & x^{17} \\8103 & \quad & x^{16}\end{matrix}$

If [d]=[d₁₅ d₁₄ . . . d₀] is the coefficient vector ofd(x)=(d₁₅x¹⁵+d₁₄x¹⁴+. . . +d₁x+d₀,), the matrix product below, [d][M],gives the coefficient vector of (d₁₅x¹⁵+d₁₄x¹⁴+. . . +d₁x+d₀)x¹⁶$\begin{matrix}{{\lbrack d\rbrack = \begin{bmatrix}0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\end{bmatrix}},{{\lbrack d\rbrack \lbrack M\rbrack} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1\end{bmatrix}}} & (8103)\end{matrix}$

$\begin{matrix}{{\lbrack d\rbrack = \begin{bmatrix}1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1\end{bmatrix}},\begin{matrix}{{\lbrack d\rbrack \lbrack M\rbrack} = \quad \begin{bmatrix}0 & 0 & 0 & 1 & 1 & 0 & 0 & 1 & 1 & 0 & 1 & 0 & 1 & 0 & 0 & 1\end{bmatrix}} \\{= \quad {\left( {{Ux}^{8} + U} \right)x^{16}}}\end{matrix}} & \left( {19{A9}} \right)\end{matrix}$

An equivalent to the matrix product is the shift_(—)16 function below:

#define X16MODG 0x8103 #define BIT15 0x8000 #define WORDMASK 0xFFFFunsigned short shift_16(d) unsigned short d; { int i; for (i=0; i<16;I++) { if(d & BIT15) { // if MSB is set, d = d << 1; // shift and xor x{circumflex over ( )}{circumflex over ( )}16, mod g(x) d = d {circumflexover ( )} X16MODG; } else d = d << 1; // otherwise, just shift } d = d &WORDMASK; return (d); } // return the d(x) * x{circumflex over ()}{circumflex over ( )}16.

A way to build a table of CRC values for all lengths of strings ofall-ones 16-bit words uses a function like shift_(—)16(d) to compute[d][M] repeatedly for d's reduced mod g(x) and augmented by 0xFFFF ineach iteration:

unsigned short ones_crc[65535]; t1build( ) { int i; unsigned short d =0; // The value at offset 0 is the CRC for for (i = 0; i < 65535; I++) {// length=1, i.e., for just 16 ones. d = d ⊕ 0xFFFF ; // If we continuedthis past 65,535 d = shift_16(d) ; // iterations, the values wouldones_crc[i] = d ; // repeat cyclically. } }

Using the same shift_(—)16 function, we may construct the antilog table.This table is just the set of d(x) * x^(16n), for n<65.535 and[d]=[0000000000000001]

unsigned short crc_antilog[65535]; genalogs( ) { unsigned d; int i; d =1; // log(1) ≡ 0. d is initialized to 1 * x^(0·16) for (i = 0; i <65535; i++) {  crc_antilog[1] = d ;  d = shift_16(d) ; // multiply byx¹⁶ again, and reduce mod g(x)  } }

The log table is built from the antilog table. We don't have any log(0),since this is −∞. We will give it special handling. (See step 1 in theprocedure of section 8.4

unsigned short crc_log[65535]; genlogs( ) { int i; for (i=0; i<65535;i++) crc_log[crc_antilog[i]] = i; }

Packet-ID  User Data  Packet Trailer Packet Header Packet Trailer(Size:) (1) (1) (4) (4) (1) (1) (1) (17) (2) (n) (4) (2) (2) (m) (2) 80Hprv block-# packet trlr comp alg 00...0 pkt-id compressed uncomp.uncomp. CRC, pad !CRC, length lgth flag # !CRC⁵ user data data user +(00...0) user + (id+user) data length CRC trlr trlr “CRC2” (1-6) (=>pad)← — — — — — — — → ← — — → — → “CRC1” “CRC3” “CRC4” ⁵The notation !CRCrepresents a CRC value calculated on an inversion of the data bytes andthen itself inverted. The Generator polynomial used in the calculationis the same one used for other CRCs in this document.

What is claimed is:
 1. A method for maintaining data integrity in astorage system that stores and retrieves data generated by a host dataprocessor, the storage system including a buffer and a controller, saidmethod comprising the steps of: receiving blocks of data from said hostcomputer; for each received block of data: forming a packet headerincluding a first group of bytes having specified functions and a header!CRC based on said predetermined bytes, with said header !CRC beingequal to a CRC calculated by inverting the bits in said first group,calculating a standard CRC on inverted bits, and inverting the standardCRC; forming a packet trailer including a second group of bytescomprised of said block and trailer bytes having specified functions,and a trailer !CRC based on a representation of said data block and saidsecond group of bytes, with said trailer !CRC being equal to a CRCcalculated by inverting all bits in said second group, calculating astandard CRC on inverted bits, and inverting the standard CRC; forming apacket by concatenating said packet header, said representation of saiddata block, and said packet trailer; subsequent to forming said packetstoring a variable number of packets in a buffer of a predeterminedsize; concatenating said variable number of packets and adding paddingbytes of all ones to form an extent having a length of a selected numberof bytes; transferring said extent to non-volatile storage through ahost adapter; in the host adapter, calculating an extent CRC for saidextent during its transfer; calculating a test CRC for a sequence ofsaid selected number of bytes of all ones; indicating a construction ortransmission error if said extent CRC and said test CRC are not thesame.
 2. The method of claim 1 wherein the block included in said packettrailer is compressed.
 3. The method of claim 1 further comprising thesteps of: storing a descriptive header in the buffer with the variablenumber of packets, the descriptive header including bytes of specifiedfunction and a !CRC based on the descriptive header; and concatenatingthe descriptive header with the variable number of packets before addingthe padding bytes.
 4. A system for error checking comprising: a channelinterface for receiving blocks of data and forming packets having aplurality of sequences with each sequence ending in a !CRC, the !CRC fora given sequence being equal to a CRC calculated by inverting the bitsin said given sequence, calculating a standard CRC on the inverted bits,and inverting the standard CRC; a first fixed-size buffer coupled to thechannel interface for storing a plurality of complete packetstransferred from said channel interface and padding bytes of all onesfilling buffer space not occupied by the complete packets, with saidplurality of complete packets and padding bytes forming an extent ofsaid fixed-size; a host adapter for transferring an extent to a storagedevice, said host adapter including: a CRC checker for calculating CRCvalues on a transmitted sequence of packets written to said storagedevice; a CRC predictor for predicting CRC predictions, based ontabulated results of dividing sequences of all ones bytes, including oneequal in length, modulo a length characteristic of the CRC generatorpolynomial, to said transmitted sequence, by a generating polynomial;means for comparing said CRC values and CRC predictions to detectconstruction or transmission errors.
 5. The system of claim 4 furthercomprising a descriptive header for the plurality of complete packets inthe first fixed-sized buffer, the descriptive header having bytes ofspecified function and a !CRC based on the bytes of the descriptiveheader.